Determining, scoring, and reporting mobile phone distraction of a driver

ABSTRACT

Among other things, information generated by sensors of a mobile phone and indicative of motion of the mobile phone and state information indicative of a state of operation of the mobile phone are monitored. Based on the monitoring, distraction by a user of the mobile phone who is a driver of a vehicle is determined.

RELATED APPLICATION

This application is a continuation, and claims the benefit of priorityunder 35 USC 120, of U.S. application Ser. No. 16/403,776, filed May 6,2019 (now allowed), the contents of which are incorporated herein intheir entirety by reference.

BACKGROUND

This description relates to determining and reporting mobile phonedistraction of a driver.

Determining and reporting mobile phone distraction of a driver is usefulbecause, among other reasons, driver mobile phone distraction poses asignificant crash and accident risk.

In an existing system developed by Cambridge Mobile Telematics servingvarious safe-driving mobile applications, sensor data from personalmobile devices (e.g., smartphones), in some cases augmented with sensordata from an optional device attached to a vehicle, has been used tomeasure the quality of driving of drivers with the goal of improvingtheir driving to make roads safer. The sensors available on a mobilephone that are useful in achieving this goal include, but are notlimited to, position sensors (e.g., the Global Positioning System, GPS),a three-axis accelerometer to measure the phone's acceleration alongthree orthogonal axes, and a three-axis gyroscope to measure the phone'sangular velocity along the three axes.

We use the term “mobile device” to include, for example, any kind ofequipment that can be carried by a user without requiring wiredconnection to a communication link and is subject to being used, while auser is driving, in a way that can cause distraction of the user fromthe driving. Mobile devices include mobile phones, for example.

In the existing system, users install a mobile application (app) on themobile phone and drive with it. The app runs in the background,automatically detecting the start and stop of each trip usinginformation from location and mobile phone activity APIs provided by theiOS and Android operating systems, in some cases augmented withinformation from the inertial sensors on the phone; or, using wirelesssignals from an in-vehicle device such as the tag device from CambridgeMobile Telematics (described in patent application publicationUS20150312655A1 and incorporated here by reference). The mobile app thengathers sensor data from the movement sensors, including positionsensors, accelerometer, and gyroscope when the user (of the phone) isdriving. This sensor data is analyzed and initially processed on themobile phone, then sent to servers in the “cloud” via a wireless network(e.g., Wi-Fi, cellular, or any other network providing connectivity tothe servers). At the cloud servers, a telematics engine processes thestream of sensor data from the mobile device to accurately estimate boththe dynamics of the vehicle and the movement patterns of the mobilephone within the vehicle. These computations could be run on the mobiledevice itself without the data being sent to cloud servers.

The estimated vehicle dynamics include map-matched positions(latitude/longitude/altitude), aspects of which are disclosed in U.S.Pat. No. 8,457,880, incorporated here by reference. The estimatedvehicle dynamics also include the “longitudinal” (in the direction ofthe vehicle, i.e., the forward acceleration and braking) and “lateral”acceleration (e.g., cornering) of the vehicle, aspects of which aredescribed in U.S. Pat. No. 9,228,836, incorporated here by reference.For braking, acceleration, and cornering, the system uses theaccelerometer and gyroscope data from the phone-which measures the forcethe phone is experiencing along three orthogonal axes, typically twoaxes parallel to the surface (e.g., the surface of the display screen)of the phone and one axis perpendicular to the surface of the mobilephone—and transforms these signals into an estimate of the accelerationof the vehicle, while ensuring that the movement of the phone relativeto the vehicle itself does not contribute. This process includesestimating and segmenting periods of time when the phone is moving in areference frame independent of the vehicle, e.g., because theorientation of the phone was changed by the user. As discussed later,the process by which this segmentation of a trip into distinct periodswhen the phone was being significantly moved within the vehicle isimportant for the assessment of phone distraction.

The servers in the cloud also compute scores for aspects of the user'sdriving, taking into account factors such as patterns of hard braking,at-risk speeding, harsh acceleration, harsh cornering, amount ofdriving, time of driving, and the user's phone distraction. In a versioncalled the DriveWell™ program, the servers also provide an overall score(typically over a rolling time window, such as over one day, two weeks,one month, three months, one year, etc.) and features to engage usersand incentivize safer driving, such as personalized driving tips, aleaderboard where users can compare their scores (to their families,friends, neighbors, co-workers, town, state, etc.), and cash prizes forsafe driving. A key aspect of the scoring is an assessment of phonedistraction.

The server also applies several classifiers to the sensor data capturedfrom the mobile device. One such classifier produces a probability orlikelihood estimate as to whether the sensor data represents data from acar or some other vehicle, such as bus, train, bike, etc. Anotherclassifier estimates, for car trips, whether the user of the device wasa passenger or driver, based on both the sensor data as well as othercontextual information, such as the start and end locations of the trip.

A “phone movement” or “phone distraction” classifier determines thelocations and times during which a user was using her phone during atrip. Typically, the location information about distraction events areprovided as a timestamped sequence of road segments, or more generally,as <start, end> latitude/longitude position tuples together withpolylines connecting the start and end positions by way of zero or moreintermediate positions (also called “waypoints”). In various versions ofthe system this feature has been termed “phone movement”, “phonemotion”, “phone use”, “phone distraction”, or “distraction”.

The existing method builds on the approach disclosed in U.S. Pat. No.9,228,836, incorporated here by reference, in which the sensor dataobtained from the trip is segmented to demarcate periods during whichthe phone was deemed to have moved relative to the frame of reference ofthe vehicle (e.g., it was moved either by a user or for other reasonswithin a vehicle). This procedure involves the use of accelerometerand/or gyroscope data; when the dynamics of these sensors is abovevarious thresholds, the phone is considered to have “moved”.

The movement of a phone does not by itself constitute “distraction”; itis, however, a factor indicative of distractiona necessary but notsufficient condition. To classify whether a particular set of segmentsof a trip with phone movement indicates distraction, the existing methodaugments the inference of phone movement with two further factors: (i)was the user likely to be interacting with the device when the phonemovement occurred, and (ii) was the vehicle moving at the time of thephone movement. A specific indicator of whether the user was interactingwith the phone uses the phone's screen state (“on” signifiesinteraction) and/or the phone's phone-lock state. With respect tovehicle movement, the existing method considers a threshold vehiclespeed below which the distraction is not considered to be occurring(e.g., a user may have pulled over and be looking at their phone ordriving slowly).

Thus, three conditions must all hold true for a distraction to beconsidered risky (and therefore a distraction episode) in the existingmethod:

-   -   A. Phone movement as inferred from gyroscope and/or        accelerometer data.    -   B. User interacting with phone, inferred particularly from        phone-screen state and phone-lock state.    -   C. The vehicle speed during the segment exceeding a threshold;        various statistics of the speed over the duration of the        distraction can be used, including the mean, median, or maximum.

The user is able to view on their mobile app's trip interface a scoredphone distraction episode. as an overlay on a map with the road segmentsshown in a different color or shade from the rest of the trajectory, andincluding some additional information (e.g., duration of distraction,speed of vehicle). In addition, by aggregating all distraction episodesfor a trip, each trip is given a phone distraction score (typically on ascale of 1 to 5 or 1 to 10). An aggregate distraction score is alsocomputed over a rolling period of time (e.g., past two weeks of driving,or any time period). FIG. 13 illustrates this interface in the existingsystem.

The existing method therefore includes a concept of episodes andseverity metrics, logging of periods of phone motion, in particular:duration of phone motion, speed of vehicle, and road type where thedistraction occurred. The existing method relies on acceleration, gyroreadings, and screen state of the mobile device, as well as accelerationfrom an external device to detect phone motion. Sensor readingsindicating movement of the mobile device must collectively last at leasta minimum duration to log a phone motion episode; and movements detectedwithin a set time interval of each other are incorporated into the sameepisode. These episodes are assessed individually to assign a risk tothe event, which is reported to users. The episodes are also assessedcollectively in generating a distraction score to display to a user: thedistraction scores are assessed per driving trip and per calendarperiod.

SUMMARY

Here we describe improvements to the existing method developed byCambridge Mobile Telematics. These enhancements include the following,among others: 1. The construction and use of “distraction episodes” toassess, score, and inform users of phone distraction occurring on atrip. 2. The use of additional factors, termed “distraction context”, toidentify user interaction with the phone in addition to screen state andphone-lock state, such as the tapping of a phone screen by the user, andprocesses to measure these additional factors. 3. The use of userfeedback to obtain information about whether a particular distractionepisode was relevant or not, for example, whether the user was apassenger, or when the user was a driver and his phone was used by apassenger in the vehicle (e.g., by a child). 4. The use of distractioncontext to score each phone distraction episode and to score each trip.

We use the term “trip” broadly to include, for example, any continuousperiod of driving within a car or other conveyance. A trip can begin ata certain starting location and time and end at a certain endinglocation and time.

We use the term “distraction episode” to include, for example, anyoccurrence, during a segment of a trajectory of a trip or during a timeperiod of a trip, of a perceptual or other disruption, confusion,diversion, or interruption of the activities of a driver in driving aconveyance.

We use the term “distraction context” broadly to include, for example,one or more circumstances or conditions associated with a distractionepisode.

We use the term “conveyance” broadly to include, for example, anyvehicle or other transportation device that can be driven on roadsduring a trip. Conveyances can include cars, trucks, carts, motorcycles,and bicycles to name a few.

In general, in an aspect, signals are received from one or moredetectors housed in a mobile device while the mobile device is on atrip. The received signals from the detectors are used to identify oneor more physical states of the mobile device during the trip. Based onthe identified physical states, discrete events separated in time aregenerated. A distraction episode is determined that spans two or more ofthe events during the trip.

Implementations may include one or a combination of two or more of thefollowing features. The detectors include an accelerometer. Thedetectors include a gyroscope. The physical state includes motion of themobile device relative to a conveyance used in the trip. The physicalstate includes an operational state of the mobile device. Theoperational state includes a state of a display screen of the mobiledevice. The physical state includes speed of motion of the mobile devicealong a trajectory of the trip. The operational state includes a stateof locking of the mobile device. The operational state includes a stateof a phone call. The operational state includes a power condition. Theoperational state includes an aspect of a foreground app running on themobile device. At least one of the discrete events includes a possiblediscrete distraction event during the trip. At least one of the discreteevents includes the mobile device being picked up. At least one of thediscrete events includes tapping on the mobile device. At least one ofthe discrete events includes the mobile device being held or facing inthe direction of a driver. At least one of the discrete events includesthe mobile device being rotated.

Determining a distraction episode includes determining a start of thedistraction episode. Determining a start of the distraction episodeincludes determining when a first physical state has begun, providedthat a second physical state occurs simultaneously with the firstphysical state within a threshold period after the first physical statebegan. Determining a distraction episode includes determining an end ofthe distraction episode. Determining an end of the distraction episodeincludes determining that a physical state of the mobile device includesno interaction by a user. Determining an end of the distraction episodeincludes determining that a physical state of the mobile device includesno further motion of the mobile device. Determining that a physicalstate of the mobile device includes no further motion of the mobiledevice includes determining that accelerometer or gyroscope values arelower than a threshold. The threshold differs from a threshold used indetermining motion of the mobile device at the start of the distractionepisode. The end of the distraction episode is not determined untilafter a refractory period of no further movement of the mobile device isno indication of a continued distraction. Determining an end of thedistraction episode includes determining that a context of the mobiledevice indicates no further distraction. Determining a distractionepisode includes determining a possible distraction episode indetermining that the possible distraction episode includes a truedistraction episode. Determining that the possible distraction episodeincludes a true distraction episode includes determining that a timeperiod spanned by the possible distraction episode is longer than apredetermined length.

The determined distraction episode is assessed. A score is assigned tothe determined distraction episode. Information about a distractionepisode is reported to a user of the mobile device or to another party.

In general, in an aspect, signals are received from one or moredetectors housed in a mobile device while the mobile device is on atrip. The received signals from the detectors are used to identify oneor more physical states of the mobile device during the trip. A possibledistraction episode is determined based on one or more of the physicalstates. A context of the possible distraction episode is determined.

Implementations may include one or a combination of two or more of thefollowing features. Determining the context of the possible distractionepisode includes: receiving signals from an operating system of themobile device or an app running on the mobile device. Based on thereceived signals, a determination is made that the mobile device isengaged in an active telephone call. Based on the received signals, adetermination is made whether the mobile device is engaged in an activetelephone call using a handset speaker, an internal speaker phone, or anexternal speaker. Based on the received signals, a proximity of a user'shand to the mobile device is determined. Based on the received signals,a determination is made whether a display screen of the mobile device isbeing tapped. Based on the received signals, a foreground app running onthe mobile device is identified. Based on the received signals, abehavior of a conveyance being used for the trip is determined.

Determining the context of a possible distraction episode includes:detecting a rotation of a display screen of the mobile device. A tap onthe display screen is determined based on a parameter of the rotation.Based on the received signals, a frequency of taps on a display screenof the mobile device is determined. Based on the received signals,temporal parameters of a sequence of taps on a display screen of themobile device are compared with temporal parameters of known sequencesof taps on display screens of mobile devices. An occurrence of a tappingevent is determined based on a tapping score based on taps in thesequence and periods of time between successive taps.

Based on the received signals, an orientation of the mobile device or adisplay screen of the mobile device during the possible distractionepisode is determined. Based on the orientation of the mobile device orthe display screen of the mobile device, the context of the possibledistraction episode is identified. When the display screen of the mobiledevice faces the back of a conveyance used for the trip, the mobiledevice is oriented generally vertically, and the mobile device has notbeen substantially rotated during at least a threshold period, thecontext of the possible distraction episode includes the mobile devicebeing mounted in the conveyance. When the display screen of the mobiledevice faces the back of a conveyance used for the trip and the mobiledevice is oriented in a direction for normal use by an occupant of aconveyance, the context of the possible distraction episode is not thatthe mobile devices mounted. When the display screen of the mobile devicefaces generally to a side of a conveyance used for the trip, and the topend of the display screen is oriented generally upwardly, the context ofthe possible distraction episode includes the mobile device being heldin the vicinity of an ear of an occupant of the conveyance. When thedisplay screen of the mobile device faces generally downwardly, thecontext of the possible distraction episode includes the mobile devicenot being used. The distraction context and the possible distractionepisode are reported to a user of the mobile device. The possibledistraction episode is scored using the distraction context. The user isshown how the user was distracted when the phone distraction episodeoccurred. The context provides an input for scoring distractionepisodes.

In general, in an aspect, signals are received from one or moredetectors housed in a mobile device while the mobile device is on atrip. A possible distraction episode is determined based on the receivedsignals. Information is received from a user who was a participant onthe trip indicative of a circumstance of the trip. A determination ismade that the possible distraction episode was a relevant distractionepisode based on the circumstances of the trip.

Implementations may include one or a combination of two or more of thefollowing features. The circumstance of the trip includes the user ofthe mobile device during the possible distraction episode not being adriver of the trip. Information about the possible distraction episodeis presented to a user. The information indicative of the circumstanceof the trip is received from the user in response to the presenting ofthe information about the possible distraction episode.

The user not being a driver of the trip is determined automatically. Theuser not being a driver of the trip is determined automatically based onprevious trips labeled by users. The determining of a possibledistraction episode includes varying the rate at which possibledistraction episodes are determined based on a parameter, and adjustingthe parameter based on information received from users about whetherpossible distraction episodes were relevant distraction episodes. Two ormore such possible distraction episodes are determined based on thereceived signals, and the possible distraction episodes for a given userare aggregated based on at least one of the following factors: types ofpossible distraction episodes, speeds associated with possibledistraction episodes, types of roads, frequencies of routes traveled, orlength of possible distraction episodes. The information about theaggregated possible distraction episodes is reported to the user.

In general, in an aspect, signals are received from one or moredetectors housed in a mobile device while the mobile devices on a trip.The received signals are indicative of a possible distraction episodeand of the context of the possible phone distraction episode. A relevantdistraction episode is determined from the possible distraction episode,and the relevant distraction episode is scored.

Implementations may include one or a combination of two or more of thefollowing features. The scoring includes accounting for the contextincluding at least one of: speed, time of day, day of week, season, roadtype, weather conditions, or direction of motion. The scoring includesaccounting for the context including at least one of movement ororientation of the mobile device or tapping on a display screen of themobile device. The result of the scoring is reported to a user. Thereporting includes showing the user a location of a relevant distractionepisode along a trajectory of the trip. The reporting includes showingthe user a score that resulted from the scoring. The reporting includesreporting to a third party. The location information for relevantdistraction episodes for two or more trips is aggregated. The contextsof relevant distraction episodes are aggregated. The aggregated contextsare reported with respect to at least one of: speed distribution duringrelevant distraction episodes, orientations of mobile devices duringrelevant distraction episodes, and activity state distributions ofmobile devices during relevant distraction episodes.

In general, in an aspect, a report is made to a user of a mobile deviceof an inference that the user was distracted by use of the mobile deviceduring a distraction episode that occurred while the user was driving.Information is received from the user indicative of whether theinference is correct. If incorrect, information is received from theuser indicative of why the inference is incorrect.

In general, in an aspect, an extent of distraction of a driver of avehicle is determined by use of a mobile device for each of two or moredistraction episodes that occurred during a trip. The extent ofdistraction of the driver during the trip is scored based on thedetermined extents and information from distraction context for thedistraction episodes.

In general, in an aspect, operation of a mobile phone is enhanced inconnection with a determination of phone distraction of a user of themobile phone while driving. For a period during a trip that has a startand an end, the mobile phone generates a set of measurementsrepresenting angular velocity or acceleration or both of the mobilephone and generates timestamps for each of the measurements, for aperiod during the trip. The mobile phone generates a set of phoneoperational states or changes in phone operational states or both andgenerates timestamps for each of the phone operational states or changesin the phone operational states. During or after the trip, at least oneof the sets of measurements, at least one of the sets of phoneoperational states, and the corresponding time stamps is provided foranalysis at the mobile phone or at a server or a combination of them, todetermine at least one phone distraction episode during the trip.

Implementations may include one or a combination of two or more of thefollowing features. The phone operational states include at least one ofa screen of the mobile phone being on, the mobile phone being unlocked,a phone call, or another state indicating a possible interaction of theuser with the mobile phone. The period includes the entire trip. Theperiod includes a moving time window with respect to the trip.

In general, in an aspect, there is monitoring of information generatedby sensors of a mobile phone and indicative of motion of the mobilephone and state information indicative of a state of operation of themobile phone. Based on the monitoring, distraction by a user of themobile phone who is a driver of a vehicle is determined.

Implementations may include one or a combination of two or more of thefollowing features. The monitoring of includes monitoring accelerationor angular velocity or both of the mobile phone. The monitoring ofinformation includes detecting movement of the mobile phone. Themonitoring of information includes determining a phone operational stateor changes of a phone operational state or both. The determining ofdistraction includes generating a phone distraction episode. Thedetermining of distraction includes determining that motion of themobile phone lasted longer than a threshold.

In general, in an aspect, determining that distraction includesdetermining that the magnitude of the motion of the mobile phone exceedsa threshold. Determining the distraction includes determining thatmotion of the phone has ended. Determining that motion of the phone hasended includes at least one of determining that the change inorientation of the mobile phone relative to gravity has fallen below athreshold; determining that the angular rotation of the mobile phone hasfallen below a threshold: determining a volatility of acceleration orangular velocity or both of the mobile phone; and determining thattapping on the screen of the phone has ended.

In general, in an aspect, there is a determination from informationabout motion of a mobile phone or a state of operation of a mobile phoneor both, that a user of the mobile phone is distracted during driving ofa vehicle, and a determination of information about at least one of anorientation of the phone, or a direction in which a display of the phoneis facing or a state of the operation of the phone information about acontext in which the user of the mobile phone was distracted duringdriving of the vehicle.

Implementations may include one or a combination of two or more of thefollowing features. The context includes at least one of: the mobilephone was mounted on the car; the user was holding the mobile phone andthe display was facing the user; and the mobile phone was being held bythe user near the user's ear. The state of operation of the mobile phoneinherently specifies the context.

In general, in an aspect, information is presented to a user of a mobilephone through a user interface of the mobile phone information aboutdistraction associated with use of the mobile phone while the user wasdriving a vehicle. The information is presented on a map showing atrajectory of a trip in which the distraction occurred. Feedback isreceived from the user about accuracy of the information presented onthe map.

In general, in an aspect, a time sequence of information about angularvelocity of the mobile device is received from the mobile device. Peaksof angular velocity along one or more axes relative to the mobile phoneare determined. A determination is made that tapping is occurring on ascreen of the mobile device based on the peaks.

Implementations may include one or a combination of two or more of thefollowing features. The peaks include related peaks that occur along twoorthogonal axes. The determining includes analyzing a frequency of theping. The determining includes comparing the detected peaks withcomparison peaks known to be associated with tapping. The states ofoperation of the mobile phone are determined based on the states ofoperation.

These and other aspects, features, and implementations can be expressedas methods, apparatus, systems, components, program products, methods ofdoing business, means or steps for performing a function, and in otherways.

These and other aspects, features, and implementations will becomeapparent from the following descriptions, including the claims.

DESCRIPTION

FIGS. 1 and 5 are block diagrams.

FIGS. 2, 6, 7, 8, and 13 are screenshots.

FIGS. 3, 4, 11, and 12 are timing graphs.

FIGS. 9, 10, and 14 are conceptual diagrams.

Among the features of and advantages achieved by various implementationsof the technology that we describe here are the following:

1. Distraction episodes: The technology aggregates an intermittentsequence of individual distraction events that may not existcontinuously—e.g., whenever the phone is determined to have moved withinthe vehicle, and the speed exceeds a threshold, and the screen is onand/or phone isn't locked and/or some form of user interaction isoccurring-into a single discrete “distraction episode”.

2. Distraction context: The technology takes advantage of new ways toassess whether a user is interacting with his phone, including steps toidentify phone tapping. Such factors constitute a “distraction context”,and are useful to assess the significance of a possible distractionepisode and to score its degree of risk accurately.

3. Incorporating user feedback: At least two additional elements ofinteraction and feedback pertaining to distraction are useful inaddition to providing a way for a user to label a trip as a “passenger”or even as a different transport mode (e.g., bus, train, etc.). In someimplementations, the first is to allow a user to dispute a flaggeddistraction episode by interacting with a map overlay of the route wherethe distraction episodes are specifically highlighted and are“clickable” for feedback to inform the system that the user did notbelieve there was a relevant distraction episode at the claimed time andlocation. The second is the common situation in which the user of thephone is in fact the driver, and the dispute is not the existence of apossible distraction episode, but whether the possible distractionepisode is a relevant distraction episode for the user as a driver,given the source of the distraction, such as a passenger in the car andnot the driver.

4. Scoring relevant distraction episodes using distraction context: Bytaking the distraction context into account when scoring a distractionepisode, the system is able to provide a score that better capturescrash risk than prior approaches to scoring phone distraction.

Some implementations of the technology enhancements that we describehere—for mobile phones, cloud servers, and systems that analyzeinformation from mobile phones—determine, among other things,distractions (e.g., distraction episodes) of drivers associated withuses of mobile phones and report information about the distractions tothe drivers (and, in some cases, other parties). The informationanalyzed and reported can include, for example, when and where thedistraction episodes occurred and other aspects of the contexts of thedistractions, including the speed, the acceleration or braking, thecornering or swerving or turning, the time-of-day, day-of-week, ormonth-of-year, the amount of prior driving on the trip, the amount ofprior driving during the preceding several hours or the same day, aswell as the nature of the user's interaction with the phone. Thedistraction context is useful in assessing a score, and is intended tocapture information indicative of the risk of a crash or accident.

Although the existing system scored distraction by estimating the totalamount of time spent distracted while driving, the technology that wedescribe here provides an approach that may better capture the risk ofan accident or crash.

In some implementations of the technology that we describe here, thedistraction is determined using data from sensors on a mobile phone. Insome cases, the technology focuses on distractions associated withholding or using the mobile phone, such as picking up a phone andtalking, using an app in ways that involve moving the phone, or tappingon the screen of the phone, typically without regard to the particularidentity of the app. In some instances the technology determines certainkinds of distraction that may occur when the mobile phone is mounted ina fixed position, or when the driver is using the phone hands-free, soas to not adversely count such occurrences toward (or otherwise toreduce their impact on) a distraction score attributable to the driver.

In some cases, the kinds of distractions of a driver that are targets ofthe technology that we describe here may be perceptual distractions thatcause a driver to divert his eyes from the road or other feature in thefield of view in the direction of driving, rather than distraction thatmay occur with, for example, hands-free calling, which may involvecognitive impact but not necessarily perceptual distraction.

In some implementations of the technology that we describe here, a basicmethod for detecting phone distraction (e.g., perceptual distraction)during driving proceeds as follows.

As shown in FIG. 1, from the start of a trip to the end of the trip (acomplete trip) or in some cases for a portion of a complete trip, phonesensor measurements 20 and states of operation 22 of the mobile phone 24(which we sometimes refer to as phone operational states) are gathered.

The measurements and states can include, for example, one or more of thefollowing timestamped data: 3-axis accelerometer readings (typicallygathered at a rate of 10 Hz or higher) 25; 3-axis gyroscope readings(typically gathered at a rate of 10 Hz or higher) 26; speed measurements(from GPS signals, typically gathered at a rate of 1 Hz or possibly fromthe vehicle's OBD interface) 30; phone barometer measurements (typicallygathered at a rate of 1 Hz or higher) 27; phone magnetometermeasurements (typically gathered at a rate of 10 Hz or higher) 29; andphone operational states 32, e.g., <time, state> or <time, new state>.The phone operational state may indicate, for example: screen on/off,device locked/unlocked, call state (e.g., “on phone”), “user activity”as defined by app events (like “launch the app” or “force quit”), theidentity and/or nature of the foreground application currently running(if any), whether the phone is plugged in or not, and other raw datauseful to compute the distraction context. Typically the phoneoperational state is recorded as a phone operational event 36 at thestart of the trip and whenever the phone operational state changes,rather than being recorded as a periodic or isochronous time series.

The process of gathering measurements and operational states can storeall measurements from the beginning of the current trip to the end ofthe trip (which may be preferable) or can maintain a circular bufferwith limited capacity to store a set of most recent measurements for amoving-window time period of a certain duration.

FIG. 3 shows an example of typical data from the 3 axes of thegyroscope; time periods when the phone's screen are on are apparent.

FIG. 4 shows an example of typical data for the GPS speed. The grayportions 90 indicate regions where the GPS speed was unavailable; whenGPS speed is unavailable, the system errs in favor of a driver beingundistracted by assuming that the vehicle may be at rest. By focusing onregions in which the speed data is available, determinations that thedriver has been distracted by his phone during those periods become evenmore credible.

Among other things, the technology described here (i) constructsdistraction episodes including a start and an end for each distractionepisode, (ii) constructs the distraction context for each distractionepisode, (iii) provides an interface to display the distraction episodeand obtain feedback from the user on a mobile app, and (iv) scores eachdistraction episode and produces an aggregate distraction score for thetrip.

Distraction Episodes

Distraction episodes are determined based on aggregations of one or moredistraction events. A single trip can have zero or more distractionepisodes. When both condition A and condition B described in theBackground section hold true, the technology determines that a potentialdistraction event has occurred; if condition C also holds, then thepotential distraction event becomes a relevant distraction event.

Because a user may move the phone only intermittently while driving, theaccelerometer and gyroscope sensor data will exhibit periods of movementinterspersed with periods of no movement often in rapid succession. Moregenerally, one or two of the three conditions A, B, and C may not holdtrue for some amount of time, as shown for example in FIG. 14 wherewithin a 30-second interval the three conditions all hold true onlyduring intermittent periods even though the driver's use of the phonewas essentially continuous through the entire 30-second period. Each ofthe intermittent periods when all three conditions hold true can beconsidered distraction events. Yet the technology avoids treating eachof them as a separate distraction episode, a treatment that would beinconsistent with the fact that the driver was effectively engaged in adistraction episode for the entire 30-second period including theintervening periods when fewer than all of the conditions held true.

The technology that we describe here provides a way to define the startand end of a distraction episode to account for such situations.

The technology is arranged to treat a distraction episode as the finestgranularity situation for which scoring is done and for whichinformation may be presented to users both for feedback and forinformational purposes. Therefore, in the situation described withrespect to FIG. 14, the technology does not score each of thedistraction events, but instead only scores the distraction episode. Forthis purpose, the technology applies certain principles, discussedbelow, to analyze the sensor signals and the timing of the occurrence ofvarious conditions in deciding that a situation should be considered adistraction episode and also in determining when such an episode startedand ended.

Sometimes, the start of a distraction episode occurs before conditions Aand B simultaneously hold true. For example, a user may pick up hisphone from a cup-holder or pocket and a few seconds later turn it on. Inthat case, the start of the distraction episode will be considered tohave occurred when one of the two conditions A and B first holds true,provided that both conditions simultaneously hold true at some timewithin a threshold period of time after the one of the two conditionsfirst holds true. Conversely, if either A or B holds true, but not both,at some point in time, and the other of A and B does not also hold truewithin the ensuing threshold period of time, then no distraction episodeis deemed to have begun. Typical values for this threshold are between 5and 30 seconds.

Similarly, a given distraction episode may not end when either conditionA or condition B no longer holds true. Instead, the end of a distractionepisode can be considered to have occurred only when all of thefollowing conditions hold true:

(i) The distraction context determination described below indicates thatthere is no further distraction (e.g., there is no more phone tapping bythe user).

(ii) The phone-movement detection process determines that there is nofurther phone movement (condition A), provided that the thresholds foraccelerometer and/or gyroscope values that result in a determination ofphone movement at the end of a distraction episode can in general bedifferent from the values to determine the start of a distractionepisode. When used, such a threshold difference can be useful because asmaller motion such as holding the phone near the ear while engaging ina phone call may justify extending the end of the distraction episoderather than terminating it.

(iii) A refractory period has elapsed during which neither of the aboveconditions (i) or (ii) holds true. A typical refractory period may be onthe order of 30 to 60 seconds. The refractory period can be adjustableusing the feedback interface provided to users.

In some implementations, a fourth condition is also required:

(iv) Condition B no longer holds true (i.e., the user is no longerinteracting with the phone).

The distraction episodes discussed to this point are considered possibledistraction episodes which may require further analysis to determine ifthey are relevant distraction episodes that should be scored.

In some implementations, a possible distraction episode must typicallylast at least a certain total amount of time to be considered a relevantdistraction episode; a too-brief possible distraction episode is oftenone not triggered by a user, and may be caused, for example, by a phonescreen being on and the vehicle driving over a pothole or speed bumpcausing the phone movement algorithm to trigger a potential event orepisode. Therefore determining phone movement is useful for this purposeand others.

To determine phone movement, the technology monitors either or both ofacceleration 104 and angular velocity 106 from the phone sensor data anduses either or both of the following techniques to determine phonemovement:

-   -   1. If accelerometer measurements are available, the method can        measure the change of gravity 110 in the phone's frame of        reference. To do this, the method maintains a sliding window of        width w around time t, including samples from t−a to t+b, where        a+b=w. The method computes two vectors, representing the        estimated direction of gravity, from each of the subwindows        [t−a, t] and [t, t+b], by summarizing acceleration values within        each window. Two typical ways of summarizing acceleration values        are (i) computing a mean vector and (ii) computing a        (component-wise) median vector. Given two “gravity” vectors, one        from the subwindow [t−a, t], g_a, and the other from the        subwindow [t, t+b], g_b, the method computes an angle between        the two vectors as arccos(g_a dot g_b/|g_a∥g_b|). If the angle        exceeds a certain threshold (between say 10 and 30 degrees, but        other values could work), the phone is considered to have        changed its orientation with respect to the direction of        gravity. This concept was described in general terms in U.S.        Pat. No. 9,228,836.    -   2. If angular velocity measurements from gyroscopes are        available, the magnitude of the phone's instantaneous rate of        rotation 112 can be computed. In some examples, the magnitude of        angular rotation rate is computed as M=sqrt(w_x{circumflex over        ( )}2+w_y{circumflex over ( )}2+w_z{circumflex over ( )}2). M is        then generally smoothed (e.g., by convolving with a Hanning        window or applying a low-pass filter). If the smoothed value of        M exceeds a certain threshold (which may be set from experience        or by learning it using a machine-learning classifier based on        feedback from users about true versus not true distraction        episodes), the phone is considered to be “rotating        significantly”. In general, this threshold may depend on the        make and model of the phone as well as being user-specific. This        concept also was described in general terms in U.S. Pat. No.        9,228,836.

Distraction Context

As noted earlier and detailed below, the scoring of a relevantdistraction episode will take into account information about thedistraction context. Although we have described rotation of the phone asone factor from which active phone use by the driver can be inferred,other factors may also be relevant to detecting active phone useincluding one or a combination of two or more of the following (derivedfrom sensors on or the operating system of) the phone: a call state thatrepresents whether the user is currently on a call (which can indicateactive phone use); whether call audio is being routed to the handsetspeaker, the internal speakerphone, or another external speaker (whichcan indicate whether the active phone use suggests perceptualdistraction); proximity of the user's hand to the phone based on datafrom a proximity sensor (such as may be present on Android phones); theidentity of the foreground app (if any) currently running on the phone(although this may be more intrusive on a user's privacy than otherfactors); whether the phone is moving in a way that is consistent withthe user tapping on the screen (e.g., typing or interacting with anapplication); whether the phone is moving in a way to suggest motion ofthe vehicle indicative of the driver possibly being distracted, e.g.,drifting behavior (as evidenced from the gyroscope signal), or drivingsubstantially below the typical speed for a road, among others.

We describe here two key examples of distraction context: (i) reliablydetecting whether a user is interactively tapping on his phone screen,and (ii) combining the phone's orientation and operational states tocharacterize distraction.

Example Distraction Context: Phone Screen Tapping and a Tapping Detector

Typing to send text messages and typing in social media apps are commonforms of phone distraction during driving. It is useful to detect thistyping behavior both to provide users with more targeted feedback abouttheir phone distraction and to assess the degree of risk (e.g., typingmay be more risky than, say, briefly swiping in a navigation app to zoominto a map). Here we describe a system for detecting typing behaviorusing a phone's sensors, without monitoring which app is running in theforeground (information that may either be unavailable or violate theuser's privacy expectations).

Typing on a mobile device typically involves tapping on a display screenof the device with the taps occurring in a pattern. Tapping can berecognized by the following process.

When a phone user holds a phone and taps the screen, the phone undergoesa small yet noticeable rotation around its x-y plane, but not muchrotation around its z-axis (FIG. 9, taken from an Android developerguide, illustrates an example arrangement of the three axes 170 relativeto the phone.)

The rate of rotation in the x-y plane, induced by a single tap on thescreen, is small (e.g., 0.2-0.7 radians/s) and lasts only for a shortperiod of time, typically under a few hundred milliseconds. The phonegyroscope captures the rotation as a curve, at a high sampling rate of,for example 100 Hz, or as a single peak at a lower frequency of, forexample, 15 Hz. The rotation is also usually followed by a revertingrotation in the reverse direction.

As shown in FIG. 10, if the screen of the phone is partitioned into fourquadrants, the quadrant on which the user taps determines the sign ofthe rotation signal along the x and y axes.

For example, if the user taps on the lower right quadrant of the screen,the phone briefly rotates along the x-axis by a positive amount andalong the y-axis also by a positive amount, assuming a right-handedcoordinate system. An example of the gyroscope signal captured by thephone when the user taps on the lower right quadrant several times isshown in FIG. 11.

As shown, when the user taps on the screen, a pair of synchronizedpeaks, one for the x-axis (solid line) and the other for y-axis(dash-dot line) appears, which is immediately followed by another peakin the reverse direction.

Another example of tapping is shown in FIG. 12, where the user taps onthe lower left quadrant. In this case, the signs of the x-axis andy-axis peaks are opposite.

Also, when the user taps on the phone in typical phone usage, such astexting or typing in a social networking page, there is a range ofplausible tapping frequencies from 5 taps a second to a few seconds pertap.

In some implementations, the tapping detector works as follows given atri-axial angular velocity measurement stream from a gyroscope in aphone:

1. Detect each tapping event by a technique that may include either orboth of the following possibilities:

-   -   a. In the input angular velocity data stream, detect occurrences        of a pair of peaks, one for the x-axis and another for the        y-axis, with peak magnitudes of the rates of angular velocity        within a range of magnitude thresholds (e.g., 0.2-0.7        radians/s). Then check if each detected pair of peaks is        followed by another pair of peaks immediately after, e.g.,        within a few tens or hundreds of milliseconds, again within a        range of thresholds.        -   i. In a “soft” thresholding version of the algorithm            (instead of hard thresholding on a fixed predefined            threshold), evaluate how much the peak magnitudes for a pair            of pair-of-peaks deviate from a typical peak range for            tapping.        -   ii. Optionally apply an additional constraint for the z-axis            (the axis perpendicular to the screen), i.e., determine            whether an angular velocity for the z-axis is below a            certain threshold.    -   b. Independently and in advance, curate a collection of actual        (known) tapping events (e.g., the occurrence of a single tap)        from test users and/or actual users. Given an input temporal        signal representing a possible tapping event, a sliding window        is applied to the signal and matched to comparable time segments        of a collection of known tapping events using, for example,        dynamic time warping. If there is a match with any of the known        tapping events, the portion of the signal within the window is        considered a tapping event. A match is defined as having a low        dynamic time warping distance between the input signal and one        or more of the signals in the collection; other definitions are        also possible.    -   c. In some implementations, the detector augments its detection        of a tapping event using the phone operational state. For        instance, the detector may not determine the existence of a        tapping event unless the screen is in an active state (e.g.,        “on” or “unlocked” or both).

2. Given a sequence of tapping events detected from step 1, the detectorcomputes a tapping score, which denotes a likelihood that the sequenceof tapping events represents the occurrence of a tapping episode.

-   -   a. The scoring starts from the zero score, S=0.    -   b. For each determined tapping event, the scorer adds the sum of        two terms, one from the tapping event itself (E) and the other        from the inter-arrival time between the respective starts of two        successive tapping events (I): S=S+(E+I).        -   i. The tapping event score, E, is a positive number that            encodes the closeness of the tapping event to a feature of a            model tapping event. For example, the absolute deviation of            the peak magnitude from the typical peak magnitude (say 0.5            radians/s) can be used, e.g., E=exp(−|M−0.5|). Or, if            tapping events were detected by matching their similarities            to tapping events of a training set, a function of the            similarity can be used.        -   ii. The inter-arrival time score, I, is evaluated in a way            to penalize inter-arrival times that are too short or too            long, which are not indicative of typical screen tapping.            For example, if the inter-arrival time is smaller than 0.1            sec or larger than 5 sec, I=−1, otherwise I=0.    -   c. If the score exceeds a threshold, a tapping episode is        determined to have occurred.    -   d. If the score falls under 0, set it to zero. That is, S=max(0,        S+(E+I)). This is to bound the effect of accumulated negative        evidence on the existence of a tapping episode.

Example Distraction Context: Combining Phone Orientation and OperationalStates

The orientation of the phone, the direction in which the phone faces,and phone operational states and combinations of them enable the systemto infer how the phone was being used when a phone distraction episodeoccurred.

Some phone operational states inherently specify a context that existedat the time of the phone distraction episode. For example, one phoneoperational state may be “active phone call”, which indicates that theuser was using his phone.

When such phone operational states are insufficient or missing and evenwhen they are available, the phone's orientation provides additionalinformation about the context of the distraction. A method fordetermining the orientation has been described in U.S. Pat. No.9,228,836, incorporated by reference. The orientation of the phone'sscreen with respect to the conveyance (such as a car) in particularreveals information about the context of phone usage at the time of aphone distraction.

In some implementations, the directions in which the phone faces and theorientations of the phone display screen and combinations of them duringphone distraction episodes can be organized into one or more of thefollowing groups (states) and others as the basis for determining aphone distraction context:

1. Mounted: The screen of the phone faces generally towards the back ofthe car and the phone is tilted (oriented) generally vertically withrespect to gravity, and the device has not experienced a rotation ofmore than a few degrees for several minutes or longer.

2. Holding with screen toward the driver: If the screen faces generallytowards the back of the car, the phone is tilted in a direction in whichpeople normally use the phone, and the phone is not in the Mountedstate.

3. Holding at ear: The screen faces generally to a side of the car, andthe top end of the screen points generally upwards (opposite of gravitydirection). This may suggest that the driver is on a phone call whileholding the phone near her ear.

4. Others: Other orientations may suggest that the driver is notactively engaged with the phone, for example, if the display screen isfacing down.

All these are inferred from the direction of the accelerometer andoptionally gyroscope vectors whose axes are in the phone's frame ofreference.

Each phone distraction episode is annotated with the inferreddistraction context, which comprises a vector of attributes capturingthe above distraction types, along with the duration of the distractionepisode and the location or time (or both) where it occurred.

The inferred distraction context can be used for at least threepurposes. First, the context can be used as a variable for furtheranalysis of phone distraction episodes, for example, to study whatdistraction contexts of phone distraction episodes are correlated highlywith the probability of a crash or near-miss. Second, the end-userinterface (eg., phone app) can present the inferred distraction contextalong with the phone distraction episode, showing to the user how theuser was distracted when the phone distraction episode occurred. Third,the context provides an input for scoring distraction episodes.

Example distraction context: weather and ambient conditions. As oneexample, the context can be the weather, including whether it is rainingor snowing, the temperature, position of the sun in the sky, etc.Certain weather conditions (e.g., snow) are known to have a much higheraccident risk, and as such distraction during such periods may be morerisky.

Example distraction context: time. As another example, distractioncontext can include time-of-day, day-of-week, and seasonal information.Certain times of day have a higher accident risk (late at night, duringrush hour), so understanding this information can be important for riskscoring purposes. Additionally, information such as how much time thedriver has already driven during this trip and during the previous 12 or24 hour period can suggest driver fatigue and reduced attentiveness,which may be relevant to the distraction score.

Example distraction context: vehicle dynamics during episode: As a thirdexample of context, it may include how the vehicle was moving during thedistraction episode, for example, whether the vehicle was slowing oraccelerating, drifting in the lane, or swerving. These behaviors canindicate the degree of distraction as well as driver inattentiveness.Finally, the relative frequency of certain driving behaviors, e.g.,harsh braking, can be compared in a given user's distracted versusundistracted periods, to measure the relative risk of distractionepisodes.

User Presentation and User Feedback

In addition to the features described above, the technology also cancollect and leverage user feedback 64 (FIG. 1) associated with a trip orone or more distraction episodes. For example, when a user looks at atrip's trajectory and notices a region or portion of the trip having aphone distraction episode, the user may indicate that the phone wasbeing used by a passenger, either for this portion of the trip or forthe whole trip. The marker for that phone distraction episode will thenbe removed from the trajectory displayed to the user or the entire tripwill be marked as a “passenger” trip, and the user will not beassociated with (penalized by) that distraction episode.

It is also useful to be able to detect automatically if a user is thedriver or passenger on a trip. Historical user-labeled trips can providea corpus of examples of passenger trips, which can be used to evaluateand improve the performance of automated driver-versus-passengerclassification. Thus, the feedback gathered as described in the previousparagraph may be used in a machine learning classifier to furtherimprove the accuracy of passenger detection.

Additionally, the user feedback can be used to improve the automaticdetection of phone distraction episodes by the technology. The parametervalues used in the steps of the process of phone distraction episodedetection need not be fixed values, but could vary within ranges. Forexample, if the settings cause over-reporting of phone distractionepisodes, this may be detected by an elevated rate of user feedbackindicating phone distraction episodes had not occurred, and used torecalibrate the system to more appropriate parameter values.

After detecting a phone distraction episode, the system may presentinformation about the episode and the phone distraction context to thedriver to help her to reduce phone distraction episodes. Sharing phonedistraction episode information also provides an opportunity to collectfeedback from the user, which can be used to validate the system and thephone distraction episodes and phone distraction contexts that itgenerates.

In some implementations, information about phone distraction episodesare reported that occur during a single trip. The system can display thetrajectory of the single trip and annotate the portions of thetrajectory during which phone distraction episodes occurred. The usercan select an individual event by tapping on it or scrolling through alist of all episodes on that trip. Selecting an episode provides apop-up with information about that episode, such as the phonedistraction context, the time it occurred, duration, and other measuredor inferred contextual information.

An example phone distraction episode is shown in FIG. 6 as a segment 140on a trajectory 142 and the time 144 of the episode is shown on a pop upthat was triggered by the user invoking the segment of the trajectory.

As shown in FIG. 7, the user then has the option of providing feedback,for example, to indicate that she was not the driver, but rather apassenger, on the trip. The user may also provide feedback that the tripwas not taken in a car, but in a bus, train, bike or other mode oftransport, or may indicate that certain portions of the labeled tripdata appear incorrectly, i.e., that they were not speeding at aparticular location.

If the user selects the link 150 called ‘Contact Support’ (orequivalent, such as ‘Provide Feedback’, which could be obtained in avariety of ways), then the user is provided with a field 152 forentering (or voicing) feedback, as shown in FIG. 8.

This feedback mechanism allows the user to correct or dispute phonedistraction episodes and the properties and context of each episode. Italso allows the user to indicate that he was the driver (i.e., the phonebelonged to the driver, not a passenger), but for one or more of thedistraction episodes in question, the distraction was due to a passengerrather than the driver.

In some implementations, the system can aggregate phone distractionepisodes and phone distraction contexts across multiple trips to providea single user-level summary. For example, the system can measure how thefrequency or count of phone distraction episodes varies by:

1. Estimated type of the phone distraction episode (phone call, texting,unlocking, for example).

2. Vehicle speed at the time of the phone distraction episode.

3. Type of road (highway vs. local road).

4. Frequency of route.

5. Length of the phone distraction episode.

Analyzing user-level phone distraction episode trends also enables thesystem to provide user-customized contextual hints for improving safety,for example, driver-specific advice like “reduce phone usage whiledriving home”, or “do not use phone at high speeds”.

Collecting user feedback allows users to feel engaged with the systemand correct potential errors in information generated by the system.Feedback from users can also be useful for validating the accuracy ofthe algorithm.

For example, a particular set of distraction parameters generated by thesystem may produce a corresponding amount of feedback. The system canuse the rate of feedback to estimate how frequently the systemmisclassifies phone distraction episodes.

By testing different parameters with users and measuring the changes inthe amount of feedback, the system can estimate the correspondingmisclassification performance for respective parameters. The system canthen modify the parameters that are generated to reduce a false positiverate or a false negative rate for those parameters.

To test different parameters, the system can change the parameters forthe whole population of users, for subsets of the population, forindividual drivers, or for individual trips, or combinations of two ormore of those approaches. Given a large population of drivers, thismakes it possible to explore the parameter space rapidly.

Instead of or in addition to providing this information to drivers, itis possible to provide it to third parties (e.g., insurance companies).

Scoring Distraction Episodes

The scoring of phone distraction episodes is done by taking thedistraction context into account. For example, distraction score candepend on speed, time-of-day, day-of-week, season, road type, weatherconditions, direction of movement, etc. In addition, the type ofdistraction can be taken into account: phone movement and light phonetapping may be treated differently in scoring. A phone call with handsetto the ear can be scored differently from a hands-free operation thatmay involve some tapping.

Uses

Phone distraction episode times and durations are typically determinedafter a trip is completed. The processes discussed above are used tomeasure that a phone distraction episode occurred; the severity of theepisode can be a function of the amount of time distracted, the speed,the road type, or the time-of-day. The distraction episode or themeasure of distraction or both can then be used in at least thefollowing ways (and others) and combinations of them:

1. Showing the user where he was distracted 52, for example, by showingthe user a map overlaid with a trajectory of her trip. Portions of thetrajectory can be marked to correspond to phone distraction episodes,that is, segments in which the phone distraction episodes were detected.These portions can be indicated by a different color in the trajectoryoverlay. This is done in the existing method.

2. Indicating each phone distraction episode using a marker overlaid ona map. Along with the marker, the context of the phone distractionepisode can be displayed, such as duration of the event, vehicle speed,road type, activity state of the phone, and the context of the trip(e.g., “on the way home”). This is done in the existing method.

3. Providing the user with a distraction score 54. The system can trackthe number of the user's phone distraction episodes over a short window(e.g., 2 weeks). This time period is chosen to be long enough to includesome averaging between trips but short enough that a dedicated drivercan see measurable improvement over the course of several days. Thisscore is incorporated in an overall score for the driver, and thisoverall score can be used to encourage competition for safe driving. Forexample, the overall scores can be sorted and displayed on aleaderboard. This is done in the existing method.

4. Providing a third party (e.g., an insurance company or a fleetmanager/operator) with a phone distraction score 56, for example, thesame score that the user sees or a score based on a longer window (e.g.,6 months) that is more appropriate for actuarial purposes. This is donein the existing method.

5. Aggregating the geographic positions of phone distraction episodesacross multiple drivers 58. This display can be useful to helpunderstand which intersections or road segments may be dangerous in acity. Such a tool is helpful for civic planners. FIG. 2 is an example ofa geographic distribution of phone distraction episodes in Miami shownby markers 60, with the color (or another visible attribute) of themarker indicating the number of phone distraction episodes at eachlocation.

6. Phone distraction profile. Various contexts of the phone distractionepisodes, including the ones described in Item 2, are aggregated andsummarized over an extended period of time (e.g., one year, or over theentire duration for which the user has been on the program). The summarycan be represented by graphical representations (plots and maps) as wellas text summaries. A few possibilities in the profile summary caninclude (but are not limited to):

-   -   a. Vehicle speed distribution while distracted, a histogram (or        equivalently a probability density plot) of vehicle speeds when        the user was distracted by phone.    -   b. Phone orientation distribution during phone distraction        episodes, a pie chart (or equivalent) showing a relative        proportion of phone orientations and usage while the user was        distracted, for example, 20% mounted, 35% holding in front, 30%        holding at ear, 15% others.    -   c. Phone activity state distribution, a pie chart (or        equivalent) showing a relative proportion of various phone        activities, for example 40% calling, 30% texting, 15% checking        navigation software, 15% unknown.

We have described technology in which the processing to determine phonedistraction episodes (and thus the display of information about thoseevents to the user or an insurance company) is done after the trip.Deferring at least the display of such information reduces the chancesof adding to driver distraction. However, in some implementations it ispossible to determine the phone distraction episodes in real time (thatis, while the trip is in progress), and to indicate to the user theoccurrence of phone distraction episodes on the phone in real time,through a visual notification (such as a pop-up message), a sound, avibration, or a combination of two or more of those indicators.

Although in some examples described above, one or more of the processesare executed on servers remote from the mobile device or on mobiledevices, the processes can be implemented on either the mobile devices,the servers, or combinations of them.

Other implementations are also within the scope of the following claims.

1-20. (canceled)
 21. A method, comprising determining, based in part onsignals received from one or more detectors of a mobile device producedwhile the mobile device is on a trip, a possible distraction episodecomprising a distraction of a user of the mobile device during the trip,the trip spanning a period longer than the possible distraction episode;after the trip, presenting information about the trip to the user of themobile device, the information about the trip including an indication ofthe possible distraction episode; after presenting the information aboutthe trip to the user of the mobile device, receiving feedback for thetrip from the user, the feedback comprising at least one of a role ofthe user of the mobile device during the trip or a mode oftransportation for the trip; and determining whether the possibledistraction episode was a relevant distraction episode based in part onthe feedback for the trip.
 22. The method of claim 21 in which theinformation about the trip presented to the user of the mobile devicecomprises a prompt for the feedback for the trip or for feedback for thepossible distraction episode, or both.
 23. The method of claim 21 inwhich the information about the trip presented to the user of the mobiledevice comprises a trajectory for the trip and the indication of thepossible distraction episode.
 24. The method of claim 23 in which thefeedback for the trip comprises an indication of at least a portion ofthe trajectory of the trip to which the feedback applies.
 25. The methodof claim 21 in which the feedback for the trip comprises an indicationof a time period during the trip to which the feedback applies.
 26. Themethod of claim 21 comprising determining that the possible distractionepisode was a relevant distraction episode when the feedback for thetrip comprises the role of the user of the mobile device being a driverof a vehicle during the trip.
 27. The method of claim 21 comprisingdetermining that the possible distraction episode was not a relevantdistraction episode when the feedback for the trip comprises the role ofthe user of the mobile device being a passenger of a vehicle during thetrip.
 28. The method of claim 21 comprising determining that thepossible distraction episode was not a relevant distraction episode whenthe feedback for the trip comprises the mode of transportation for thetrip being a non-car mode.
 29. The method of claim 21 in which the tripcomprises a first trip, the method comprising: determining whether asecond possible distraction episode comprising a distraction of the userof the mobile device during a second trip different from the first trip,was a relevant distraction episode based in part on the feedback fromthe user for the first trip.
 30. The method of claim 21 comprisinggenerating a score for the trip or the possible distraction episode, orboth, based in part on the received signals and the feedback for thetrip.
 31. The method of claim 21 in which determining the possibledistraction episode comprises comparing the received signals with one ormore parameters, the method comprising adjusting at least one of the oneor more parameters based on the feedback from the user for the trip. 32.The method of claim 21 in which the possible distraction episodecomprises two or more discrete distraction events separated in time. 33.The method of claim 21 in which each of the discrete distraction eventscomprises concurrent motion of the mobile device and interaction withthe mobile device by the user.
 34. A system, comprising: one or moreprocessors; and storage for instructions executable by the one or moreprocessors to: determine, based in part on signals received from one ormore detectors of a mobile device produced while the mobile device is ona trip, a possible distraction episode comprising a distraction of auser of the mobile device during the trip, the trip spanning a periodlonger than the possible distraction episode; after the trip, presentinformation about the trip to the user of the mobile device, theinformation about the trip including an indication of the possibledistraction episode; after presenting the information about the trip tothe user of the mobile device, receive feedback for the trip from theuser, the feedback comprising at least one of a role of the user of themobile device during the trip or a mode of transportation for the trip;and determine whether the possible distraction episode was a relevantdistraction episode based in part on the feedback for the trip.
 35. Thesystem of claim 34 in which the feedback for the trip comprises anindication of a time period during the trip to which the feedbackapplies or at least a portion of a trajectory of the trip to which thefeedback applies.
 36. The system of claim 34 in which the one or moreprocessors determine that the possible distraction episode was arelevant distraction episode when the feedback for the trip comprisesthe role of the user of the mobile device being a driver of a vehicleduring the trip.
 37. The system of claim 34 in which the one or moreprocessors determine that the possible distraction episode was not arelevant distraction episode when the feedback for the trip comprisesthe role of the user of the mobile device being a passenger of a vehicleduring the trip or the mode of transportation for the trip being anon-car mode.
 38. A non-transitory computer-readable storage mediumhaving instructions which, when executed by one or more processors,cause the one or more processors to perform operations comprising:determining, based in part on signals received from one or moredetectors of a mobile device produced while the mobile device is on atrip, a possible distraction episode comprising a distraction of a userof the mobile device during the trip, the trip spanning a period longerthan the possible distraction episode; after the trip, presentinginformation about the trip to the user of the mobile device, theinformation about the trip including an indication of the possibledistraction episode; after presenting the information about the trip tothe user of the mobile device, receiving feedback for the trip from theuser, the feedback comprising at least one of a role of the user of themobile device during the trip or a mode of transportation for the trip;and determining whether the possible distraction episode was a relevantdistraction episode based in part on the feedback for the trip.
 39. Thenon-transitory computer-readable medium of claim 38 comprisingdetermining that the possible distraction episode was a relevantdistraction episode when the feedback for the trip comprises the role ofthe user of the mobile device being a driver of a vehicle during thetrip.
 40. The non-transitory computer-readable medium of claim 38comprising determining that the possible distraction episode was not arelevant distraction episode when the feedback for the trip comprisesthe role of the user of the mobile device being a passenger of a vehicleduring the trip or the mode of transportation for the trip being anon-car mode.